Blocked microphone detector and wind sock detector

ABSTRACT

An image capture device performs audio processing based on a detection of whether a wind sock is attached to the image capture device. When a wind sock is detected, the image capture device performs stereo processing. When a wind sock is not detected, the image capture device performs wind processing.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to and the benefit of U.S. ProvisionalApplication Patent Ser. No. 63/359,139, filed Jul. 7, 2022, the entiredisclosure of which is hereby incorporated by reference.

TECHNICAL FIELD

This disclosure relates to audio processing. In particular, thisdisclosure relates to audio processing and detection of blockedmicrophones and/or wind socks.

BACKGROUND

Audio processing in image capture devices can be negatively affected byvarious factors, including blocked microphone ports and wind noise.Image capture devices typically have microphone ports that can beunintentionally blocked by a finger of a user when using the imagecapture device by hand. A blocked microphone port can interrupt audioprocessing and disrupt a natural audio experience when audio signals areencoded. In a case of wind noise, wind noise can affect differentialmicrophone arrays when beamforming and can prevent stereo and/orbeamforming processing.

SUMMARY

Disclosed herein are implementations of an image capture deviceconfigured to detect an attached wind sock. The image capture deviceincludes a microphone, a sensor, and a processor. The microphone may beconfigured to detect audio signals. The sensor may be configured todetect whether a wind sock is attached to the image capture device. Theprocessor may be configured to perform stereo processing of the audiosignals based on a detection of an attached wind sock to obtainprocessed audio signals. The processor may be configured to perform windprocessing based on a non-detection of an attached wind sock to obtainprocessed audio signals. The processor may be configured to output theprocessed audio signals.

A method for wind sock detection may include detecting audio signalsusing one or more microphones of an image capture device. The method mayinclude determining whether a wind sock is attached to the image capturedevice. The method may include prioritizing stereo processing over windprocessing based on a determination that the wind sock is attached. Themethod may include prioritizing wind processing over stereo processingbased on a determination that the wind sock is not attached.

A method for blocked microphone detection may include receiving audiosignals from at least two microphones. An algorithm to detect a blockedmicrophone may be trained using machine learning. The method may includesplitting the audio signals into frequency sub-bands. The method mayinclude applying, in each frequency sub-band, an amplitude offset basedon a noise floor. The method may include determining, in each frequencysub-band, a first correlation metric between offset audio signals fromthe at least two microphones. The method may include calculating asecond correlation metric from frequency sub-bands below a firstthreshold frequency. The method may include determining whether thesecond correlation metric is below a second threshold. The method mayinclude determining that a microphone of the at least two microphones isunblocked based on a determination that the second correlation metric isabove the second threshold. The method may include determining that amicrophone of the at least two microphones is blocked based on adetermination that the second correlation metric is below the secondthreshold. The method may include transmitting a notification thatindicates whether a microphone of the at least two microphones isblocked or unblocked.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is best understood from the following detaileddescription when read in conjunction with the accompanying drawings. Itis emphasized that, according to common practice, the various featuresof the drawings are not to-scale. On the contrary, the dimensions of thevarious features are arbitrarily expanded or reduced for clarity.

FIGS. 1A-B are isometric views of an example of an image capture device.

FIGS. 2A-B are isometric views of another example of an image capturedevice.

FIG. 2C is a top view of the image capture device of FIGS. 2A-B.

FIG. 2D is a partial cross-sectional view of the image capture device ofFIG. 2C.

FIG. 3 is a block diagram of electronic components of an image capturedevice.

FIG. 4 is a block diagram of an example of a wind sock configured foruse with the image capture devices shown in FIGS. 1A to 3 .

FIG. 5 is a flow diagram of an example of a method for performing audioprocessing with wind sock detection.

FIG. 6 is a flow diagram of an example of a method for detecting ablocked microphone.

FIG. 7 is a flow diagram of another example of a method for detecting ablocked microphone.

DETAILED DESCRIPTION

Many users of image capture devices do not have an understanding of thepositioning of the microphone ports on their image capture devices.Accordingly, these users may be unaware that blocking just onemicrophone port can negatively affect beamforming used for stereoprocessing. The users may also be unaware of how to hold the imagecapture device so that the microphone ports are unblocked. This lack ofawareness may be an issue when the user is attempting to hold the imagecapture device and use it in a vlogging or point-of-view (POV) use casewithout an attached grip. If the user repositions a hand in the wrongway, the user may accidentally trigger audio processing that isinappropriate for the given scene/scenario. The implementationsdescribed herein may provide a visual notification, a hapticnotification, an audible notification, or any combination thereof to theuser to identify such a condition. For example, an audible alert can beused when the user cannot see a display of the image capture device,such as when the display is oriented away from the user. In thisexample, the user may receive an audible notification, such as a voicenotification or other audible notification that a microphone port isblocked.

Typical audio processing detects the presence of wind and activelyswitches to a wind processing mode. This wind processing mode activelyreduces wind noise recorded by the camera, however, stereo processing islost. In some situations, vibrations from handling the image capturedevice can falsely trigger a wind processing event when there is no windpresent. The automated switching between wind processing and stereomodes can also be distracting to the user and result in a poor userexperience. A wind sock can be used to minimize turbulent noise causedby wind. However, with a wind sock present, the image capture device maystill be susceptible to falsely detecting wind when vibration isexperienced. The implementations described herein include an imagecapture device that is configured to detect the presence of a wind socksuch that when a wind sock is detected, the image capture device isconfigured to prioritize stereo processing to avoid false winddetections and unneeded changes to wind processing mode due to handling.In some examples, with a wind sock present, some true detection of windmay occur. In these examples, with the knowledge of an attached windsock, a different type of wind processing (e.g., other than stereoprocessing) may be performed.

FIGS. 1A-B are isometric views of an example of an image capture device100. The image capture device 100 may include a body 102, a lens 104structured on a front surface of the body 102, various indicators on thefront surface of the body 102 (such as light-emitting diodes (LEDs),displays, and the like), various input mechanisms (such as buttons,switches, and/or touch-screens), and electronics (such as imagingelectronics, power electronics, etc.) internal to the body 102 forcapturing images via the lens 104 and/or performing other functions. Thelens 104 is configured to receive light incident upon the lens 104 andto direct received light onto an image sensor internal to the body 102.The image capture device 100 may be configured to capture images andvideo and to store captured images and video for subsequent display orplayback.

The image capture device 100 may include an LED or another form ofindicator 106 to indicate a status of the image capture device 100 and aliquid-crystal display (LCD) or other form of a display 108 to showstatus information such as battery life, camera mode, elapsed time, andthe like. The image capture device 100 may also include a mode button110 and a shutter button 112 that are configured to allow a user of theimage capture device 100 to interact with the image capture device 100.For example, the mode button 110 and the shutter button 112 may be usedto turn the image capture device 100 on and off, scroll through modesand settings, and select modes and change settings. The image capturedevice 100 may include additional buttons or interfaces (not shown) tosupport and/or control additional functionality.

The image capture device 100 may include a door 114 coupled to the body102, for example, using a hinge mechanism 116. The door 114 may besecured to the body 102 using a latch mechanism 118 that releasablyengages the body 102 at a position generally opposite the hingemechanism 116. The door 114 may also include a seal 120 and a batteryinterface 122. When the door 114 is an open position, access is providedto an input-output (I/O) interface 124 for connecting to orcommunicating with external devices as described below and to a batteryreceptacle 126 for placement and replacement of a battery (not shown).The battery receptacle 126 includes operative connections (not shown)for power transfer between the battery and the image capture device 100.When the door 114 is in a closed position, the seal 120 engages a flange(not shown) or other interface to provide an environmental seal, and thebattery interface 122 engages the battery to secure the battery in thebattery receptacle 126. The door 114 can also have a removed position(not shown) where the entire door 114 is separated from the imagecapture device 100, that is, where both the hinge mechanism 116 and thelatch mechanism 118 are decoupled from the body 102 to allow the door114 to be removed from the image capture device 100.

The image capture device 100 may include a microphone 128 on a frontsurface and another microphone 130 on a side surface. The image capturedevice 100 may include other microphones on other surfaces (not shown).The microphones 128, 130 may be configured to receive and record audiosignals in conjunction with recording video or separate from recordingof video. The image capture device 100 may include a speaker 132 on abottom surface of the image capture device 100. The image capture device100 may include other speakers on other surfaces (not shown). Thespeaker 132 may be configured to play back recorded audio or emit soundsassociated with notifications.

A front surface of the image capture device 100 may include a drainagechannel 134. A bottom surface of the image capture device 100 mayinclude an interconnect mechanism 136 for connecting the image capturedevice 100 to a handle grip or other securing device. In the exampleshown in FIG. 1B, the interconnect mechanism 136 includes foldingprotrusions configured to move between a nested or collapsed position asshown and an extended or open position (not shown) that facilitatescoupling of the protrusions to mating protrusions of other devices suchas handle grips, mounts, clips, or like devices.

The image capture device 100 may include an interactive display 138 thatallows for interaction with the image capture device 100 whilesimultaneously displaying information on a surface of the image capturedevice 100.

The image capture device 100 of FIGS. 1A-B includes an exterior thatencompasses and protects internal electronics. In the present example,the exterior includes six surfaces (i.e. a front face, a left face, aright face, a back face, a top face, and a bottom face) that form arectangular cuboid. Furthermore, both the front and rear surfaces of theimage capture device 100 are rectangular. In other embodiments, theexterior may have a different shape. The image capture device 100 may bemade of a rigid material such as plastic, aluminum, steel, orfiberglass. The image capture device 100 may include features other thanthose described here. For example, the image capture device 100 mayinclude additional buttons or different interface features, such asinterchangeable lenses, cold shoes, and hot shoes that can addfunctional features to the image capture device 100.

The image capture device 100 may include various types of image sensors,such as charge-coupled device (CCD) sensors, active pixel sensors (APS),complementary metal-oxide-semiconductor (CMOS) sensors, N-typemetal-oxide-semiconductor (NMOS) sensors, and/or any other image sensoror combination of image sensors.

Although not illustrated, in various embodiments, the image capturedevice 100 may include other additional electrical components (e.g., animage processor, camera system-on-chip (SoC), etc.), which may beincluded on one or more circuit boards within the body 102 of the imagecapture device 100.

The image capture device 100 may interface with or communicate with anexternal device, such as an external user interface device (not shown),via a wired or wireless computing communication link (e.g., the I/Ointerface 124). Any number of computing communication links may be used.The computing communication link may be a direct computing communicationlink or an indirect computing communication link, such as a linkincluding another device or a network, such as the internet, may beused.

In some implementations, the computing communication link may be a Wi-Filink, an infrared link, a Bluetooth (BT) link, a cellular link, a ZigBeelink, a near field communications (NFC) link, such as an ISO/IEC 20643protocol link, an Advanced Network Technology interoperability (ANT+)link, and/or any other wireless communications link or combination oflinks.

In some implementations, the computing communication link may be an HDMIlink, a USB link, a digital video interface link, a display portinterface link, such as a Video Electronics Standards Association (VESA)digital display interface link, an Ethernet link, a Thunderbolt link,and/or other wired computing communication link.

The image capture device 100 may transmit images, such as panoramicimages, or portions thereof, to the external user interface device viathe computing communication link, and the external user interface devicemay store, process, display, or a combination thereof the panoramicimages.

The external user interface device may be a computing device, such as asmartphone, a tablet computer, a phablet, a smart watch, a portablecomputer, personal computing device, and/or another device orcombination of devices configured to receive user input, communicateinformation with the image capture device 100 via the computingcommunication link, or receive user input and communicate informationwith the image capture device 100 via the computing communication link.

The external user interface device may display, or otherwise present,content, such as images or video, acquired by the image capture device100. For example, a display of the external user interface device may bea viewport into the three-dimensional space represented by the panoramicimages or video captured or created by the image capture device 100.

The external user interface device may communicate information, such asmetadata, to the image capture device 100. For example, the externaluser interface device may send orientation information of the externaluser interface device with respect to a defined coordinate system to theimage capture device 100, such that the image capture device 100 maydetermine an orientation of the external user interface device relativeto the image capture device 100.

Based on the determined orientation, the image capture device 100 mayidentify a portion of the panoramic images or video captured by theimage capture device 100 for the image capture device 100 to send to theexternal user interface device for presentation as the viewport. In someimplementations, based on the determined orientation, the image capturedevice 100 may determine the location of the external user interfacedevice and/or the dimensions for viewing of a portion of the panoramicimages or video.

The external user interface device may implement or execute one or moreapplications to manage or control the image capture device 100. Forexample, the external user interface device may include an applicationfor controlling camera configuration, video acquisition, video display,or any other configurable or controllable aspect of the image capturedevice 100.

The user interface device, such as via an application, may generate andshare, such as via a cloud-based or social media service, one or moreimages, or short video clips, such as in response to user input. In someimplementations, the external user interface device, such as via anapplication, may remotely control the image capture device 100 such asin response to user input.

The external user interface device, such as via an application, maydisplay unprocessed or minimally processed images or video captured bythe image capture device 100 contemporaneously with capturing the imagesor video by the image capture device 100, such as for shot framing orlive preview, and which may be performed in response to user input. Insome implementations, the external user interface device, such as via anapplication, may mark one or more key moments contemporaneously withcapturing the images or video by the image capture device 100, such aswith a tag or highlight in response to a user input or user gesture.

The external user interface device, such as via an application, maydisplay or otherwise present marks or tags associated with images orvideo, such as in response to user input. For example, marks may bepresented in a camera roll application for location review and/orplayback of video highlights.

The external user interface device, such as via an application, maywirelessly control camera software, hardware, or both. For example, theexternal user interface device may include a web-based graphicalinterface accessible by a user for selecting a live or previouslyrecorded video stream from the image capture device 100 for display onthe external user interface device.

The external user interface device may receive information indicating auser setting, such as an image resolution setting (e.g., 3840 pixels by2160 pixels), a frame rate setting (e.g., 60 frames per second (fps)), alocation setting, and/or a context setting, which may indicate anactivity, such as mountain biking, in response to user input, and maycommunicate the settings, or related information, to the image capturedevice 100.

The image capture device 100 may be used to implement some or all of thetechniques and methods described in this disclosure, such as the method500 described in FIG. 5 or the method 600 described in FIG. 6 .

FIGS. 2A-B illustrate another example of an image capture device 200.The image capture device 200 includes a body 202 and two camera lenses204 and 206 disposed on opposing surfaces of the body 202, for example,in a back-to-back configuration, Janus configuration, or offset Janusconfiguration. The body 202 of the image capture device 200 may be madeof a rigid material such as plastic, aluminum, steel, or fiberglass.

The image capture device 200 includes various indicators on the front ofthe surface of the body 202 (such as LEDs, displays, and the like),various input mechanisms (such as buttons, switches, and touch-screenmechanisms), and electronics (e.g., imaging electronics, powerelectronics, etc.) internal to the body 202 that are configured tosupport image capture via the two camera lenses 204 and 206 and/orperform other imaging functions.

The image capture device 200 includes various indicators, for example,LEDs 208, 210 to indicate a status of the image capture device 100. Theimage capture device 200 may include a mode button 212 and a shutterbutton 214 configured to allow a user of the image capture device 200 tointeract with the image capture device 200, to turn the image capturedevice 200 on, and to otherwise configure the operating mode of theimage capture device 200. It should be appreciated, however, that, inalternate embodiments, the image capture device 200 may includeadditional buttons or inputs to support and/or control additionalfunctionality.

The image capture device 200 may include an interconnect mechanism 216for connecting the image capture device 200 to a handle grip or othersecuring device. In the example shown in FIGS. 2A and 2B, theinterconnect mechanism 216 includes folding protrusions configured tomove between a nested or collapsed position (not shown) and an extendedor open position as shown that facilitates coupling of the protrusionsto mating protrusions of other devices such as handle grips, mounts,clips, or like devices.

The image capture device 200 may include audio components 218, 220, 222such as microphones configured to receive and record audio signals(e.g., voice or other audio commands) in conjunction with recordingvideo. The audio component 218, 220, 222 can also be configured to playback audio signals or provide notifications or alerts, for example,using speakers. Placement of the audio components 218, 220, 222 may beon one or more of several surfaces of the image capture device 200. Inthe example of FIGS. 2A and 2B, the image capture device 200 includesthree audio components 218, 220, 222, with the audio component 218 on afront surface, the audio component 220 on a side surface, and the audiocomponent 222 on a back surface of the image capture device 200. Othernumbers and configurations for the audio components are also possible.

The image capture device 200 may include an interactive display 224 thatallows for interaction with the image capture device 200 whilesimultaneously displaying information on a surface of the image capturedevice 200. The interactive display 224 may include an I/O interface,receive touch inputs, display image information during video capture,and/or provide status information to a user. The status informationprovided by the interactive display 224 may include battery power level,memory card capacity, time elapsed for a recorded video, etc.

The image capture device 200 may include a release mechanism 225 thatreceives a user input to in order to change a position of a door (notshown) of the image capture device 200. The release mechanism 225 may beused to open the door (not shown) in order to access a battery, abattery receptacle, an I/O interface, a memory card interface, etc. (notshown) that are similar to components described in respect to the imagecapture device 100 of FIGS. 1A and 1B.

In some embodiments, the image capture device 200 described hereinincludes features other than those described. For example, instead ofthe I/O interface and the interactive display 224, the image capturedevice 200 may include additional interfaces or different interfacefeatures. For example, the image capture device 200 may includeadditional buttons or different interface features, such asinterchangeable lenses, cold shoes, and hot shoes that can addfunctional features to the image capture device 200.

FIG. 2C is a top view of the image capture device 200 of FIGS. 2A-B andFIG. 2D is a partial cross-sectional view of the image capture device200 of FIG. 2C. The image capture device 200 is configured to capturespherical images, and accordingly, includes a first image capture device226 and a second image capture device 228. The first image capturedevice 226 defines a first field-of-view 230 and includes the lens 204that receives and directs light onto a first image sensor 232.Similarly, the second image capture device 228 defines a secondfield-of-view 234 and includes the lens 206 that receives and directslight onto a second image sensor 236. To facilitate the capture ofspherical images, the image capture devices 226 and 228 (and relatedcomponents) may be arranged in a back-to-back (Janus) configuration suchthat the lenses 204, 206 face in generally opposite directions.

The fields-of-view 230, 234 of the lenses 204, 206 are shown above andbelow boundaries 238, 240 indicated in dotted line. Behind the firstlens 204, the first image sensor 232 may capture a firsthyper-hemispherical image plane from light entering the first lens 204,and behind the second lens 206, the second image sensor 236 may capturea second hyper-hemispherical image plane from light entering the secondlens 206.

One or more areas, such as blind spots 242, 244 may be outside of thefields-of-view 230, 234 of the lenses 204, 206 so as to define a “deadzone.” In the dead zone, light may be obscured from the lenses 204, 206and the corresponding image sensors 232, 236, and content in the blindspots 242, 244 may be omitted from capture. In some implementations, theimage capture devices 226, 228 may be configured to minimize the blindspots 242, 244.

The fields-of-view 230, 234 may overlap. Stitch points 246, 248 proximalto the image capture device 200, that is, locations at which thefields-of-view 230, 234 overlap, may be referred to herein as overlappoints or stitch points. Content captured by the respective lenses 204,206 that is distal to the stitch points 246, 248 may overlap.

Images contemporaneously captured by the respective image sensors 232,236 may be combined to form a combined image. Generating a combinedimage may include correlating the overlapping regions captured by therespective image sensors 232, 236, aligning the captured fields-of-view230, 234, and stitching the images together to form a cohesive combinedimage.

A slight change in the alignment, such as position and/or tilt, of thelenses 204, 206, the image sensors 232, 236, or both, may change therelative positions of their respective fields-of-view 230, 234 and thelocations of the stitch points 246, 248. A change in alignment mayaffect the size of the blind spots 242, 244, which may include changingthe size of the blind spots 242, 244 unequally.

Incomplete or inaccurate information indicating the alignment of theimage capture devices 226, 228, such as the locations of the stitchpoints 246, 248, may decrease the accuracy, efficiency, or both ofgenerating a combined image. In some implementations, the image capturedevice 200 may maintain information indicating the location andorientation of the lenses 204, 206 and the image sensors 232, 236 suchthat the fields-of-view 230, 234, the stitch points 246, 248, or bothmay be accurately determined; the maintained information may improve theaccuracy, efficiency, or both of generating a combined image.

The lenses 204, 206 may be laterally offset from each other, may beoff-center from a central axis of the image capture device 200, or maybe laterally offset and off-center from the central axis. As compared toimage capture devices with back-to-back lenses, such as lenses alignedalong the same axis, image capture devices including laterally offsetlenses may include substantially reduced thickness relative to thelengths of the lens barrels securing the lenses. For example, theoverall thickness of the image capture device 200 may be close to thelength of a single lens barrel as opposed to twice the length of asingle lens barrel as in a back-to-back lens configuration. Reducing thelateral distance between the lenses 204, 206 may improve the overlap inthe fields-of-view 230, 234. In another embodiment (not shown), thelenses 204, 206 may be aligned along a common imaging axis.

Images or frames captured by the image capture devices 226, 228 may becombined, merged, or stitched together to produce a combined image, suchas a spherical or panoramic image, which may be an equirectangularplanar image. In some implementations, generating a combined image mayinclude use of techniques including noise reduction, tone mapping, whitebalancing, or other image correction. In some implementations, pixelsalong the stitch boundary may be matched accurately to minimize boundarydiscontinuities.

The image capture device 200 may be used to implement some or all of thetechniques and methods described in this disclosure, such as the method500 described in FIG. 500 or the method 600 described in FIG. 600 .

FIG. 3 is a block diagram of electronic components in an image capturedevice 300. The image capture device 300 may be a single-lens imagecapture device, a multi-lens image capture device, or variationsthereof, including an image capture device with multiple capabilitiessuch as use of interchangeable integrated sensor lens assemblies. Thedescription of the image capture device 300 is also applicable to theimage capture devices 100, 200 of FIGS. 1A-B and 2A-D.

The image capture device 300 includes a body 302 which includeselectronic components such as capture components 310, a processingapparatus 320, data interface components 330, movement sensors 340,power components 350, and/or user interface components 360.

The capture components 310 include one or more image sensors 312 forcapturing images and one or more microphones 314 for capturing audio.

The image sensor(s) 312 is configured to detect light of a certainspectrum (e.g., the visible spectrum or the infrared spectrum) andconvey information constituting an image as electrical signals (e.g.,analog or digital signals). The image sensor(s) 312 detects lightincident through a lens coupled or connected to the body 302. The imagesensor(s) 312 may be any suitable type of image sensor, such as acharge-coupled device (CCD) sensor, active pixel sensor (APS),complementary metal-oxide-semiconductor (CMOS) sensor, N-typemetal-oxide-semiconductor (NMOS) sensor, and/or any other image sensoror combination of image sensors. Image signals from the image sensor(s)312 may be passed to other electronic components of the image capturedevice 300 via a bus 380, such as to the processing apparatus 320. Insome implementations, the image sensor(s) 312 includes adigital-to-analog converter. A multi-lens variation of the image capturedevice 300 can include multiple image sensors 312.

The microphone(s) 314 is configured to detect sound, which may berecorded in conjunction with capturing images to form a video. Themicrophone(s) 314 may also detect sound in order to receive audiblecommands to control the image capture device 300.

The processing apparatus 320 may be configured to perform image signalprocessing (e.g., filtering, tone mapping, stitching, and/or encoding)to generate output images based on image data from the image sensor(s)312. The processing apparatus 320 may include one or more processorshaving single or multiple processing cores. In some implementations, theprocessing apparatus 320 may include an application specific integratedcircuit (ASIC). For example, the processing apparatus 320 may include acustom image signal processor. The processing apparatus 320 may exchangedata (e.g., image data) with other components of the image capturedevice 300, such as the image sensor(s) 312, via the bus 380.

The processing apparatus 320 may include memory, such as a random-accessmemory (RAM) device, flash memory, or another suitable type of storagedevice, such as a non-transitory computer-readable memory. The memory ofthe processing apparatus 320 may include executable instructions anddata that can be accessed by one or more processors of the processingapparatus 320. For example, the processing apparatus 320 may include oneor more dynamic random-access memory (DRAM) modules, such as double datarate synchronous dynamic random-access memory (DDR SDRAM). In someimplementations, the processing apparatus 320 may include a digitalsignal processor (DSP). More than one processing apparatus may also bepresent or associated with the image capture device 300.

The data interface components 330 enable communication between the imagecapture device 300 and other electronic devices, such as a remotecontrol, a smartphone, a tablet computer, a laptop computer, a desktopcomputer, or a storage device. For example, the data interfacecomponents 330 may be used to receive commands to operate the imagecapture device 300, transfer image data to other electronic devices,and/or transfer other signals or information to and from the imagecapture device 300. The data interface components 330 may be configuredfor wired and/or wireless communication. For example, the data interfacecomponents 330 may include an I/O interface 332 that provides wiredcommunication for the image capture device, which may be a USB interface(e.g., USB type-C), a high-definition multimedia interface (HDMI), or aFireWire interface. The data interface components 330 may include awireless data interface 334 that provides wireless communication for theimage capture device 300, such as a Bluetooth interface, a ZigBeeinterface, and/or a Wi-Fi interface. The data interface components 330may include a storage interface 336, such as a memory card slotconfigured to receive and operatively couple to a storage device (e.g.,a memory card) for data transfer with the image capture device 300(e.g., for storing captured images and/or recorded audio and video).

The movement sensors 340 may detect the position and movement of theimage capture device 300. The movement sensors 340 may include aposition sensor 342, an accelerometer 344, or a gyroscope 346. Theposition sensor 342, such as a global positioning system (GPS) sensor,is used to determine a position of the image capture device 300. Theaccelerometer 344, such as a three-axis accelerometer, measures linearmotion (e.g., linear acceleration) of the image capture device 300. Thegyroscope 346, such as a three-axis gyroscope, measures rotationalmotion (e.g., rate of rotation) of the image capture device 300. Othertypes of movement sensors 340 may also be present or associated with theimage capture device 300.

The power components 350 may receive, store, and/or provide power foroperating the image capture device 300. The power components 350 mayinclude a battery interface 352 and a battery 354. The battery interface352 operatively couples to the battery 354, for example, with conductivecontacts to transfer power from the battery 354 to the other electroniccomponents of the image capture device 300. The power components 350 mayalso include an external interface 356, and the power components 350may, via the external interface 356, receive power from an externalsource, such as a wall plug or external battery, for operating the imagecapture device 300 and/or charging the battery 354 of the image capturedevice 300. In some implementations, the external interface 356 may bethe I/O interface 332. In such an implementation, the I/O interface 332may enable the power components 350 to receive power from an externalsource over a wired data interface component (e.g., a USB type-C cable).

The user interface components 360 may allow the user to interact withthe image capture device 300, for example, providing outputs to the userand receiving inputs from the user. The user interface components 360may include visual output components 362 to visually communicateinformation and/or present captured images to the user. The visualoutput components 362 may include one or more lights 364 and/or moredisplays 366. The display(s) 366 may be configured as a touch screenthat receives inputs from the user. The user interface components 360may also include one or more speakers 368. The speaker(s) 368 canfunction as an audio output component that audibly communicatesinformation and/or presents recorded audio to the user. The userinterface components 360 may also include one or more physical inputinterfaces 370 that are physically manipulated by the user to provideinput to the image capture device 300. The physical input interfaces 370may, for example, be configured as buttons, toggles, or switches. Theuser interface components 360 may also be considered to include themicrophone(s) 314, as indicated in dotted line, and the microphone(s)314 may function to receive audio inputs from the user, such as voicecommands.

The image capture device 300 may be used to implement some or all of thetechniques and methods described in this disclosure, such as the method500 described in FIG. 5 or the method 600 described in FIG. 6 .

FIG. 4 is a block diagram of an example of a wind sock 402 configuredfor use with an image capture device 404. The image capture device 404may be an image capture device such as the image capture device 300shown in FIG. 3 .

As shown in FIG. 4 , the image capture device 404 includes a detectioninterface 406 and a sensor 408. The detection interface 406 may includean interface for magnetic detection, switch detection, photo sensordetection, or another detection interface. The sensor 408 may include ahall effect sensor, a switch, a photo sensor, or another sensor. Thesensor 408 is configured to detect the presence of the wind sock 402.The sensor 408 may transmit an indication of the presence of the windsock 402 to a processor, such as processing apparatus 320 shown in FIG.3 .

The wind sock 402 includes one or more magnets 410. The one or moremagnets 410 are configured to attach the wind sock 402 to the detectioninterface 406 of the image capture device 404. In some embodiments, amagnet of the one or more magnets 410 may be replaced with an actuatorthat is configured to activate a switch on the detection interface 406of the image capture device 404. The sensor 408 is configured to detectwhen the wind sock 402 is attached and transmit an indication of thepresence of the wind sock 402 to the processor. The processor isconfigured to receive the indication of the presence of the wind sock402 and prioritize beamforming processing over wind processing.Accordingly, the processor is configured to perform beamformingprocessing in response to receiving an indication of the presence of thewind sock 402. Though described as using one or more magnets 410 tocouple the wind sock 402 and the image capture device 404, othermechanical connection mechanisms, such as hooks, latches, detents,clips, slots, etc. may be used to couple the wind sock 402 and the imagecapture device 404.

FIG. 5 is a flow diagram of an example of a method 500 for performingaudio processing with wind sock detection. The method 500 may beperformed by an image capture device, such as the image capture device404 shown in FIG. 4 .

The method 500 includes detecting 502 audio signals. The audio signalsmay be detected 502 using one or more microphones, such as the one ormore microphones 314 shown in FIG. 3 .

The method 500 includes determining 504 whether a wind sock is attachedto the image capture device. The determination of whether a wind sock isattached may be based on sensor data. The sensor data may include anindication of whether the wind sock is attached to the image capturedevice. The sensor data may include data associated with a magneticsensor (e.g., a hall effect sensor), data associated with a switch, dataassociated with a photo sensor, or data associated with another sensor.

Based on a determination that a wind sock is attached, the method 500includes performing 506 stereo processing or another processing. Themethod may prioritize stereo processing or another processing over windprocessing based on the determination that the wind sock is attached.The other processing may include, for example, deactivating waterdetection to save processing power since a wind sock is not used inwater use cases, or applying an alternate calibration for themicrophones with the wind sock attached. Based on a determination thatthe wind sock is not attached, the method 500 includes performing 508wind processing. The method may prioritize wind processing over stereoprocessing based on the determination that the wind sock is notattached.

FIG. 6 is a flow diagram of an example of a method 600 for detecting ablocked microphone. In this example, the image capture device may have 3microphones. The 3 microphones may include a front microphone (e.g.,channel 1 (Ch1)), a side microphone (e.g., Ch2), and a top microphone(e.g., Ch3). Example blocked states of the microphones may be defined asshown in Table 1 below, where a 0 indicates an unblocked state and a 1indicates a blocked state.

TABLE 1 Ch1 (Front) Ch2 (Side) Ch3 (Top) Blocked Case Output 0 0 0 0Operate as normal 0 1 0 1 Alert user the side microphone is blocked 0 01 2 Alert user the top microphone is blocked 0 1 1 3 Alert user the sideand top microphone are blocked 1 0 0 4 Alert user the front microphoneis blocked 1 0 1 5 Alert user the front and top microphones are blocked1 1 0 6 Alert user the front and side microphones are blocked 1 1 1 7Alert user the front, side, and top microphones are blocked

Referring to FIG. 6 , audio signals are received 602 from at least twomicrophones, which may include two of the microphones 314 shown in FIG.3 . The audio signals may comprise audio blocks for a particular timeinterval of a longer audio stream, such as one-half second, one second,five seconds, or another duration. The audio signals are each split 604in a plurality of different frequency sub-bands. In each frequencysub-band, a noise-floor dependent amplitude offset is applied 606. Thenoise floor represents, for each sub-band, a threshold amplitude levelof a minimum amount of noise expected to be present in the audio signal.The respective noise floors for different sub-bands may be different. Agreater amplitude offset is applied in sub-bands with higher noisefloors to ensure that the signals can be reliably correlated in thefollowing steps. In each sub-band, a sub-band correlation metric isdetermined 608 between the offset audio signals from the two or moremicrophones. The sub-band correlation metric may represent a similaritybetween signal levels of audio block sub-bands captured by themicrophones for a given time interval. Generally, the signals will bewell-correlated in the absence of wind noise or other noise havingsimilar noise profiles in the given sub-band, but will be poorlycorrelated when wind noise or other noise is present in the sub-band.Thus, the correlation metric may operate as a wind detection metric. Inone embodiment, each sub-band correlation metric comprises a value from0 to 1 where a correlation metric of 1 represents a situation consistentwith little to no uncorrelated noise present in the sub-band, and acorrelation metric of 0 means that the captured audio may besubstantially comprised of uncorrelated noise such as wind noise.

An overall correlation metric is calculated 610 for all sub-bands belowa frequency threshold (e.g., below 1500 Hz). The overall correlationmetric may comprise for example, an average (e.g., mean) or weightedaverage of the sub-band correlation metrics for sub-bands under thefrequency threshold. The overall correlation metric is compared 612 to apredefined threshold. In one embodiment, the predefined threshold maydynamically change between two or more predefined thresholds dependingon the previous state (e.g., whether the threshold was exceeded in theprevious audio block) to include a hysteresis effect. For example, iffor the previously processed block, the correlation metric exceeded thepredefined threshold (e.g., a predefined threshold of 0.8 or 0.9), thenthe predefined threshold is set lower for the current block (e.g., 0.6or 0.7). If for the previously processed block, the correlation metricdid not exceed the predefined threshold (e.g., a predefined threshold of0.8 or 0.9), then the predefined threshold for the current block is sethigher (e.g., to 0.85 or 0.95).

If the correlation metric exceeds the predefined threshold in step 612,a determination 614 a determination is made that at least one of the oneor more of the microphones is unblocked. For example, in one embodiment,a correlated audio signal processing algorithm can be used to generate acombined audio signal based on blockage conditions associated with eachof the microphones (e.g., whether each microphone is blocked orunblocked). If it is determined that at least one or more of themicrophones is unblocked, the method 600 returns to operation 602. Insome implementations, a notification may be transmitted 618 thatindicates that at least one of the one or more microphones is unblocked.If the overall correlation metric is below the threshold in step 612, adetermination 616 is made that at least one of the one or moremicrophones 314 is blocked. In an example, the uncorrelated processingalgorithm may select, for each frequency band, a frequency component ofan audio signal having the lowest uncorrelated noise and combine thesefrequency components together to create the combined audio signal.

A notification is transmitted 618 that indicates that at least one ofthe one or more microphones is blocked. The notification may betransmitted during video capture or prior to video capture, such as whenthe user is framing the subject of the video capture. In an example, thenotification may alert the user to alter their hand position and uncoverthe microphone, for example, using audible beeps, voice alerts, hapticalerts, or any combination thereof. In some embodiments, the method mayinclude switching to an unblocked microphone for audio capture orapplying compensation processing to the unblocked microphone to reducethe effect of the blocked microphone. In some embodiments, the detectionof a blocked microphone can be used to influence other image capturedevice processing. For example, the image capture device may update anelectronic image stabilization algorithm based on a determination thatthe image capture device is handheld (i.e., based on a determinationthat a microphone is blocked).

In some embodiments, machine learning may be used to train an algorithmto detect a blocked microphone. In some embodiments, the detection ofuncorrelated audio signals in certain frequency ranges may be used todetermine whether a microphone is blocked or unblocked. In someembodiments, a detection of whether a microphone is blocked or unblockedmay use a speaker to play a tone and analyzing the microphone signal todetect an expected frequency response of a blocked or unblockedmicrophone. This may be performed during an image capture device idlestate. In some examples, the tone may be outside the audible range toavoid interfering with the audio capture.

FIG. 7 is a flow diagram of another example of a method 700 fordetecting a blocked microphone. In this example, machine learning may beused to train an algorithm to detect a blocked microphone.

The method 700 includes obtaining 702 audio data. The audio data may beobtained using one or more microphones, such as the one or moremicrophones 314 shown in FIG. 3 .

The method 700 includes selecting 704 a number of frequency bins. Eachfrequency bin may be a 93.75 Hz bin. The selected frequency bins mayvary from 0-3000 Hz. In a 3 microphone image capture device example, 9frequency bins may be selected, and each selected frequency bin may bedifferent.

The method 700 includes performing 706 complex squares and complexmagnitude computations for each selected frequency bin.

The method 700 includes generating 708 transfer functions to obtainfeature vectors. The transfer functions may be generated by a divisionbetween the magnitude of one block and the square magnitude of anotherblock. The transfer functions may be generated ratios of second orderstatistics. An example of a transfer function may be a complex multiplybetween channel 1 and channel 2 divided by the complex square ofchannel 1. For an image capture device that has 3 microphones, a featurevector may be generated by concatenating 6 transfer function estimatesat 9 different frequency bins. In an example, each frequency bin may bea 93.75 Hz bin. The selected 9 bins for finger detection (e.g., blockedstate detection) may vary from 0-3000 Hz. The feature vector may use thefollowing transfer function estimates:

$\frac{ch1 \times ch2}{\left( {ch1} \right)^{2}};\frac{ch1 \times ch2}{\left( {ch2} \right)^{2}};\frac{ch1 \times ch3}{\left( {ch1} \right)^{2}};\frac{ch1 \times ch3}{\left( {ch3} \right)^{2}};\frac{ch2 \times ch3}{\left( {ch2} \right)^{2}};\frac{ch2 \times ch3}{\left( {ch3} \right)^{2}}$

For training and validation, separate data sets for each covered statemay be obtained. The data sets may be classified and run through theaudio algorithm front end to generate the feature vectors.

The method 700 includes training 710 a model using the feature vectors.The model may be a multinomial logistic regression model. Training themodel may include sampling data blocks (e.g., where each data block isapproximately 5 ms) to determine whether the current blocked/unblockedstate is constant for a predetermined time, such as 1 second, forexample. If the current blocked/unblocked state is constant for thepredetermined time, the current blocked/unblocked state is confirmed anda switch to the current blocked/unblocked state may be made. If thecurrent blocked/unblocked state is not constant for the predeterminedtime, the blocked/unblocked state remains at the previous state.

The method 700 includes obtaining 712 learned model coefficients. Thelearned model coefficients may be derived from the determinedblocked/unblocked states.

The method 700 includes updating 714 an audio algorithm using thelearned model coefficients. In an implementation, the model coefficientsare learned offline and then applied back to the audio algorithm. Thelogistic regression model uses the coefficients to compute theprobability of the blocked state in real-time.

While the disclosure has been described in connection with certainembodiments, it is to be understood that the disclosure is not to belimited to the disclosed embodiments but, on the contrary, is intendedto cover various modifications and equivalent arrangements includedwithin the scope of the appended claims, which scope is to be accordedthe broadest interpretation so as to encompass all such modificationsand equivalent structures as is permitted under the law.

What is claimed is:
 1. An image capture device, comprising: a microphoneconfigured to detect audio signals; a sensor configured to detectwhether a wind sock is attached to the image capture device; and aprocessor configured to: perform stereo processing of the audio signalsbased on a detection of an attached wind sock to obtain processed audiosignals; perform wind processing of the audio signals based on anon-detection of an attached wind sock to obtain processed audiosignals; and output the processed audio signals.
 2. The image capturedevice of claim 1, wherein the sensor is a hall effect sensor, a switch,or a photo sensor.
 3. The image capture device of claim 1, furthercomprising: a detection interface that is configured to attach to thewind sock.
 4. The image capture device of claim 3, wherein the detectioninterface is configured to attach to the wind sock magnetically.
 5. Theimage capture device of claim 1, wherein the sensor is furtherconfigured to transmit an indication of a presence of the wind sock tothe processor.
 6. The image capture device of claim 1, wherein the windsock comprises one or more magnets.
 7. The image capture device of claim1, wherein the wind sock comprises an actuator configured to activate aswitch on a detection interface of the image capture device.
 8. A methodfor wind sock detection, comprising: detecting audio signals using oneor more microphones of an image capture device; determining whether awind sock is attached to the image capture device based on sensor data;prioritizing stereo processing over wind processing based on adetermination that the wind sock is attached; and prioritizing windprocessing over stereo processing based on a determination that the windsock is not attached.
 9. The method of claim 8, wherein determiningwhether a wind sock is attached is based on sensor data.
 10. The methodof claim 9, wherein the sensor data includes an indication of whetherthe wind sock is attached to the image capture device.
 11. The method ofclaim 9, wherein the sensor data includes data associated with a halleffect sensor, data associated with a switch, or data associated with aphoto sensor.
 12. The method of claim 8, further comprising: performingstereo processing based on a determination that the wind sock isattached.
 13. The method of claim 8, further comprising: performing windprocessing based on a determination that the wind sock is not attached.14. A method, comprising: receiving audio signals from at least twomicrophones; splitting the audio signals into frequency sub-bands;applying, in each frequency sub-band, an amplitude offset based on anoise floor; determining, in each frequency sub-band, a firstcorrelation metric between offset audio signals from the at least twomicrophones; calculating a second correlation metric from frequencysub-bands below a first threshold frequency; determining whether thesecond correlation metric is below a second threshold; determining thata microphone of the at least two microphones is unblocked based on adetermination that the second correlation metric is above the secondthreshold; determining that a microphone of the at least two microphonesis blocked based on a determination that the second correlation metricis below the second threshold; and transmitting a notification thatindicates whether a microphone of the at least two microphones isblocked or unblocked.
 15. The method of claim 14, wherein thenotification is a visual notification, an audible notification, or ahaptic notification.
 16. The method of claim 14, wherein the audiosignals comprise audio blocks for a time interval of an audio stream.17. The method of claim 14, wherein the noise floor is a thresholdamplitude level of a minimum amount of noise expected to be present inthe audio signals.
 18. The method of claim 14, wherein the firstcorrelation metric represents a similarity between signal levels ofsub-bands captured by the at least two microphones for a time interval.19. The method of claim 14, wherein the first correlation metric is avalue between 0 and
 1. 20. The method of claim 14, wherein the secondcorrelation metric is an average of the first correlation metric foreach sub-band below the first threshold frequency.